Increasing research interests focus on sequential recommender systems, aiming to model dynamic sequence representation precisely. However, the most commonly used loss function in state-of-the-art sequential recommendation models has essential limitations. To name a few, Bayesian Personalized Ranking (BPR) loss suffers the vanishing gradient problem from numerous negative sampling and predictionbiases; Binary Cross-Entropy (BCE) loss subjects to negative sampling numbers, thereby it is likely to ignore valuable negative examples and reduce the training efficiency; Cross-Entropy (CE) loss only focuses on the last timestamp of the training sequence, which causes low utilization of sequence information and results in inferior user sequence representation. To avoid these limitations, in this paper, we propose to calculate Cumulative Cross-Entropy (CCE) loss over the sequence. CCE is simple and direct, which enjoys the virtues of painless deployment, no negative sampling, and effective and efficient training. We conduct extensive experiments on five benchmark datasets to demonstrate the effectiveness and efficiency of CCE. The results show that employing CCE loss on three state-of-the-art models GRU4Rec, SASRec, and S3-Rec can reach 125.63%, 69.90%, and 33.24% average improvement of full ranking NDCG@5, respectively. Using CCE, the performance curve of the models on the test data increases rapidly with the wall clock time, and is superior to that of other loss functions in almost the whole process of model training.

在现实世界中的问题回答场景中，将表格和文本内容均结合的混合形式吸引了越来越多的关注，其中数值推理问题是最典型和最具挑战性的问题之一。现有方法通常采用编码器框架来表示混合内容并生成答案。但是，它无法捕获编码器侧数值，表格架构和文本信息之间的丰富关系。解码器使用一个简单的预定义运算符分类器，该分类器的灵活性不足以处理具有不同表达式的数值推理过程。为了解决这些问题，本文提出了一个\ textbf {re} lational \ textbf {g} raph增强\ textbf {h} ybrid table-text \ textbf {n}带有\ textbf {t textbf {t text} ree decoder（\ textbff recoder（\ textbf） {reghnt}）。它模拟了对表 - 文本混合内容的回答的数值问题，作为表达树的生成任务。此外，我们提出了一种新颖的关系图建模方法，该方法模拟了问题，表和段落之间的对齐方式。我们验证了公开可用的Table-Text混合质量质量质量标准（TAT-QA）的模型。拟议的reghnt显着胜过基线模型，并实现最新结果\脚注{我们在〜\ url {https://github.com/lfy79001/reghnt}}}〜（20222）公开发布了源代码和数据-05-05）。

网络安全漏洞是分布式网络物理系统（CPS）的常见异常。但是，即使使用尖端人工智能（AI）方法，网络安全漏洞分类仍然是一个困难的问题。在本文中，我们研究了网络安全性的多类分类问题，以进行攻击检测。考虑了一个具有挑战性的多节点数据审查案例。在这种情况下，当本地数据不完整时，每个数据中心/节点中的数据都无法共享。特别是，本地节点仅包含多个类别的一部分。为了培训全球多级分类器而不在所有节点上共享原始数据，我们研究的主要结果是设计多节点多级分类合奏方法。通过从每个局部节点收集二进制分类器和数据密度的估计参数，每个局部节点的丢失信息都可以完成，以构建全局多类分类器。进行数值实验以验证在多节点数据审查情况下提出的方法的有效性。在这种情况下，我们甚至显示了对全数据ATA方法的拟议方法的表现。

亚当是训练深神经网络的最具影响力的自适应随机算法之一，即使在简单的凸面设置中，它也被指出是不同的。许多尝试，例如降低自适应学习率，采用较大的批量大小，结合了时间去相关技术，寻求类似的替代物，\ textit {etc。}，以促进Adam-type算法融合。与现有方法相反，我们引入了另一种易于检查的替代条件，这仅取决于基础学习率的参数和历史二阶时刻的组合，以确保通用ADAM的全球融合以解决大型融合。缩放非凸随机优化。这种观察结果以及这种足够的条件，对亚当的差异产生了更深刻的解释。另一方面，在实践中，无需任何理论保证，广泛使用了迷你ADAM和分布式ADAM。我们进一步分析了分布式系统中的批次大小或节点的数量如何影响亚当的收敛性，从理论上讲，这表明迷你批次和分布式亚当可以通过使用较大的迷你批量或较大的大小来线性地加速节点的数量。最后，我们应用了通用的Adam和Mini Batch Adam，具有足够条件来求解反例并在各种真实世界数据集上训练多个神经网络。实验结果完全符合我们的理论分析。

Data Augmentation (DA) is frequently used to automatically provide additional training data without extra human annotation. However, data augmentation may introduce noisy data that impairs training. To guarantee the quality of augmented data, existing methods either assume no noise exists in the augmented data and adopt consistency training or use simple heuristics such as training loss and diversity constraints to filter out ``noisy'' data. However, those filtered examples may still contain useful information, and dropping them completely causes loss of supervision signals. In this paper, based on the assumption that the original dataset is cleaner than the augmented data, we propose an on-the-fly denoising technique for data augmentation that learns from soft augmented labels provided by an organic teacher model trained on the cleaner original data. A simple self-regularization module is applied to force the model prediction to be consistent across two distinct dropouts to further prevent overfitting on noisy labels. Our method can be applied to augmentation techniques in general and can consistently improve the performance on both text classification and question-answering tasks.

Visual language such as charts and plots is ubiquitous in the human world. Comprehending plots and charts requires strong reasoning skills. Prior state-of-the-art (SOTA) models require at least tens of thousands of training examples and their reasoning capabilities are still much limited, especially on complex human-written queries. This paper presents the first one-shot solution to visual language reasoning. We decompose the challenge of visual language reasoning into two steps: (1) plot-to-text translation, and (2) reasoning over the translated text. The key in this method is a modality conversion module, named as DePlot, which translates the image of a plot or chart to a linearized table. The output of DePlot can then be directly used to prompt a pretrained large language model (LLM), exploiting the few-shot reasoning capabilities of LLMs. To obtain DePlot, we standardize the plot-to-table task by establishing unified task formats and metrics, and train DePlot end-to-end on this task. DePlot can then be used off-the-shelf together with LLMs in a plug-and-play fashion. Compared with a SOTA model finetuned on more than >28k data points, DePlot+LLM with just one-shot prompting achieves a 24.0% improvement over finetuned SOTA on human-written queries from the task of chart QA.

Visual language data such as plots, charts, and infographics are ubiquitous in the human world. However, state-of-the-art vision-language models do not perform well on these data. We propose MatCha (Math reasoning and Chart derendering pretraining) to enhance visual language models' capabilities in jointly modeling charts/plots and language data. Specifically, we propose several pretraining tasks that cover plot deconstruction and numerical reasoning which are the key capabilities in visual language modeling. We perform the MatCha pretraining starting from Pix2Struct, a recently proposed image-to-text visual language model. On standard benchmarks such as PlotQA and ChartQA, the MatCha model outperforms state-of-the-art methods by as much as nearly 20%. We also examine how well MatCha pretraining transfers to domains such as screenshots, textbook diagrams, and document figures and observe overall improvement, verifying the usefulness of MatCha pretraining on broader visual language tasks.

End-to-end (E2E) task-oriented dialogue (ToD) systems are prone to fall into the so-called 'likelihood trap', resulting in generated responses which are dull, repetitive, and often inconsistent with dialogue history. Comparing ranked lists of multiple generated responses against the 'gold response' (from training data) reveals a wide diversity in response quality, with many good responses placed lower in the ranked list. The main challenge, addressed in this work, is then how to reach beyond greedily generated system responses, that is, how to obtain and select such high-quality responses from the list of overgenerated responses at inference without availability of the gold response. To this end, we propose a simple yet effective reranking method which aims to select high-quality items from the lists of responses initially overgenerated by the system. The idea is to use any sequence-level (similarity) scoring function to divide the semantic space of responses into high-scoring versus low-scoring partitions. At training, the high-scoring partition comprises all generated responses whose similarity to the gold response is higher than the similarity of the greedy response to the gold response. At inference, the aim is to estimate the probability that each overgenerated response belongs to the high-scoring partition, given only previous dialogue history. We validate the robustness and versatility of our proposed method on the standard MultiWOZ dataset: our methods improve a state-of-the-art E2E ToD system by 2.4 BLEU, 3.2 ROUGE, and 2.8 METEOR scores, achieving new peak results. Additional experiments on the BiTOD dataset and human evaluation further ascertain the generalisability and effectiveness of the proposed framework.

在原始文本中训练的语言模型（LMS）无法直接访问物理世界。 Gordon和Van Durme（2013）指出，LMS因此可能会遭受报告偏见的困扰：文本很少报告常见事实，而是关注情况的异常方面。如果LMS仅接受文本语料库的培训，并天真地记住当地的同时出现统计数据，那么他们自然会学会对物理世界的偏见。虽然先前的研究反复验证了较小尺度的LM（例如Roberta，GPT-2）放大了报告偏差，但在模型扩展时，这种趋势是否继续。我们从较大语言模型（LLM）（例如Palm和GPT-3）中从颜色的角度研究报告偏见。具体而言，我们查询llms对物体的典型颜色，这是一种简单的感知扎根的物理常识。令人惊讶的是，我们发现LLM在确定对象的典型颜色和更紧密地跟踪人类判断方面的表现明显优于较小的LMS，而不是过于适应文本中存储的表面图案。这表明，仅凭语言的大型语言模型就能克服以局部共发生为特征的某些类型的报告偏差。

我们引入了一种新的文化学习范式，以测量在推理过程中学习新颖单词的大型语言模型（LLMS）。特别是，我们通过用一个合成但合理的词代替关键概念词来重写Winograd风格的共同参考分辨率问题，该词必须理解该模型以完成任务。解决此任务需要模型来利用提示中给出的新单词的字典定义。这个基准介绍了单词获取，这是折磨llms已知的历时降解的一个重要方面。由于LLM在训练的那一刻及时被冻结，因此通常无法反映语言随着时间的变化方式。我们表明，与原始Winograd任务相比，LLM的准确性在我们的基准测试中从根本上降低，从而确定了当前模型的局限性，并提供了基准来衡量LLMS的未来改善LLMS进行内在学习的能力。